feat(doc-index): infer real section granularity, hash sections for staleness - #1

…abric#109

- cmd_doc_index() built its output from the index but omitted
  retrieval_sections/section_level in both JSON and human output -- the
  new data constructorfabric#109 added was invisible through the CLI. Both are exposed now,
  and the human formatter lists retrieval sections the same way it already
  lists the finer-grained ones.
- A write landing between read_text() and _compute_etag() in
  build_doc_index() could save headings parsed from the *old* content
  stamped with the *new* file's etag; load_doc_index() would then treat
  that stale index as valid until a later edit changed the etag again.
  _read_with_stable_etag() brackets the read with a stat snapshot on each
  side and retries on mismatch, so the saved etag is provably the one that
  matches what was actually parsed.
- diff_stale_sections() reported changed/unchanged sections by heading
  text alone; two sections sharing a duplicate title (a real,
  already-flagged possibility -- see toc-heading-duplicate) couldn't be
  told apart. Each entry now carries line_start alongside the heading
  text, which is what a caller should actually use to address "this
  specific section" afterwards.
- annotate_section_summary() updated only index["sections"], leaving the
  matching retrieval_sections entry at summary=None even on success -- a
  caller reading retrieval_sections (the more relevant list for a future
  per-section summarizer) couldn't see the annotation. Now updates both
  when both have an entry at line_start.
- toc.py's _frontmatter_has_description() accepted `description: # TODO`
  and `description: ""` as satisfying the check, since `#` and `"` both
  match \S. Now parses the field's actual value and rejects comments and
  empty/whitespace-only quoted strings.

Extracted _compute_fresh_retrieval_sections/_position_entry out of
diff_stale_sections() to stay under pylint's local-variable limit after
the line_start addition; registered the new instructions (stable-read,
diff-stale-helpers) in traceability-validation.md.

See constructorfabric#104.

Verified: pytest (test_doc_index.py + test_toc.py: 166 passed, 100%
coverage on touched doc_index files); full suite: 4825 passed, the same
12 pre-existing macOS-local/flaky failures as on constructorfabric#108/constructorfabric#109, none in files
touched here; pylint and vulture clean; cfs validate 0 errors;
spec-coverage thresholds met; infer_section_level/retrieval_sections
re-verified against the real PDF-converted document that originally
exposed the granularity bug -- still 12 correct sections.

Signed-off-by: TECK KEAT WILSON <yeow.teck.keat@constructor.tech>

…torfabric#109

- load_doc_index() returned a cached index whenever its etag matched,
  with no check that the cached shape matched what this version of the
  code expects. A cache written before section_level/retrieval_sections
  existed can still have a matching etag if the file hasn't changed since
  -- cmd_doc_index() would then hit a KeyError reading those fields on a
  legacy cache instead of a clean rebuild. Now treated the same as a
  stale cache: rebuilt, not returned as-is.
- _frontmatter_has_description() treated a YAML block-scalar marker
  (`description: |`, `description: >-`, ...) as a usable value, when the
  real content -- if any -- belongs on indented lines below it, not on
  the marker's own line. Now checks the first non-blank following line
  for real indentation before counting it as a description.

See constructorfabric#104.

Verified: pytest (test_doc_index.py + test_toc.py: 172 passed, 100%
coverage on touched doc_index files); full suite: 4831 passed, the same
12 pre-existing macOS-local/flaky failures as before, none in files
touched here; pylint and vulture clean; cfs validate 0 errors;
spec-coverage thresholds met; infer_section_level/retrieval_sections
re-verified against the real PDF-converted document -- still 12 correct
sections.

Signed-off-by: TECK KEAT WILSON <yeow.teck.keat@constructor.tech>